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IDENTIFIEBS 
ABSTBACT 

The power of a statistical test is, in part, a 
function of the reliability of the dependabli variable being 
analyzed. The substitution of sigma square divided by the reliability 
coeffxcxent for sigma is proposed. This enables the researcher to 
incorporate dependent variable reliability information when 
determining the sample size required for a specified power of his 
statistical test. An inverse relationship between test reliability 
and the sample n required to maintain a given statistical power is 
defined. (Author) 
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EDUCATJDN POS»TtON OR POLICY 

We know that the reliability of the dependent variable measure is related 
to power (the ability to reject a Pa/j^null hypothesis). Kerlinger (1964), 
Helmstadter (1970), and others have reminded us that increased error variance 
in a dependent variable will increase the likelihood of Type II errors. However, ' 
we fail to find in the literature a formula that will enable us to systematically 
integrate reliability information into power computations. 

The purpose of this paper is to propose a way in which knowledge of depen- 
dent variable reliability can be used in a typical two-group comparison of 
means analysis to enable a researcher to more accurately determine the sample 
size requited for a specified power of his statistic. 

Let us consider the formula for determining the number of subjects (n) - 
required for rejecting given an absolute difference between population means 
(S ), level of significance ( «< ) , proportion of replications in which H will 
be rejected (power or 1 - R ), and population variance ( cr 2) as given . by 
Edwards (1967): ^ & / 

£. = 2_£_ (2<^- + )2 Formula 1 

Note that this formula does net incorporate the reliability of the instru- 
ment used to measure the dependent variable. The formula will yield the same 
n for a dependent variable measure with a variance of 100 and reliability of 
.40 as it does for a test with a variance of 100 and a reliability of .95. 

We know that if dependent variable variance is kept constant, an inves- 
tigator employing a measure with low reliability has less ability to reject 
a false null hypothesis (power) than an° investigator employing a measure with 
high reliability. We also know that in making comparisons between groups we 
can compensate for low reliability through increasing the number in the groups. 

We propose to demonstrate that we can systematically establish how much 
increase in n is required to compensate for a decrease in reliability by employ- 
ing the definitions of true variance and error variance in such a way that as 
error variance increases, an appropriate compensating increase in n will be 
indicated. n % ~ 

The observed population variance can be partialled into true and error 
variance components using classical variance formulas from Gullikson, 1950: 
Observed variance = True variance + Error variance; True variance ^ Observed 
variance = test reliability; therefore. True variance = Observed variance x' 
test reliability. Whenever reliability is less than perfect, observed variance 
will be greater than true variance and the observed standard deviation (cr ) 
will be greater than the true standard deviation (<r ). *^ 

Delta ( ^ ) is by definition a true difference between population means 
^/*r/'.J' we define % as a distance in true standard deviation units (CZ^ ) 
rather than observed standard deviation units (C^ ), we can establish s systematic 
method for adjusting n according to reliability. We find no authority which 



2 



maintains that S must be defined in true standard deviation units* On the 
.other hand we find no authority which maintains that it should be defined in 
observed standard deviation units. So we shall precede from the premise that 
the former is reasonable and define S > a true difference, measured in true 
standard deviation units (<roo ) rather than measured in observed standard 
deviation units ( ^ ) which are a combination of true and error variation. 

( 2 = <5^, 2 ^ 2j 

_ If we define the " distance in cr;^ units, the variance of 

Xj - X2 under a true ( /^.^ = 0) tjf^mes: ^ — 
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and when the ratio 




J . • 

equals unity solving for n yields a formula identical to formula 1 except 
for the substitution of C^o ^ for cr . 

When an instrument with less than perfect reliability is employed, 
error variance is added to produce observed variance equal to the true 
variance divided by reliability. 
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The expected observed variance of Xj - X2 becomes: .j^^ 

2(<^ 4- )2 2c%o ^ 

^ QI- : 

./Jl nr 

and a null hypotheses at c=< level will be rejected proportion of trials 
when the ratio: 

equals unity. Solving this equation for n we have 
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As a result of the amendment in the numerator of the formula, the size - 
of n. will be larger when claculated by Formula 2 than when calculated with 
Formula 1, except when reliability is perfect (r^ « 1.0). To illustrate 
this modification, consider an example first computed with the original ^ 
Formula 1, then with the amended Formula 2, using two different reliability 
coefficients. 

An investigator has a treatment which is hypothesized to increase WISC 
full scale IQ scores. He can randomly assign subjects to E and C groups. 
He wants a true mean difference of one^third the standard deviation to be 
statistically significant at the .05 level in 90% of replications of the 
experiment. Given the WISC standard deviation of 15, he computes numbers of 
subjects required using Formula 1 as follows; 



n = 



2(15)^ ^ 450 



(1.96 + 1,28) 25 (10.5) 189.0 



Employing Formula 1 the investigator would conclude that he needs 189 
subjects in each of his two groups to obtain the power desired. However, 
if we define £ in true standard deviation units this would be the case 
only if the reliability of the WISC is unity (which it is not). 

Now assume the experiment is to be done with 10% year olds for whom 
the Wechsler (1949) manual reports a reliability of .95. Using the amended 
Formula 2, we have the following: 

2 

n = 2|15)^ ^ J 28)^ . A50^ (10.5) - JMA 

5 (.95) 23.75 

Even with this very high reliability, the number required in each of the two 
groups is about 10 more than the number derived with the standard Formula 1. 

A further increase in n will bo required if the experiment is done with 
7h year olds, for whom the Wechsler (1949) manual reports a reliability of .92. 

Again, using Formula 2, we have the following: 

- ^ I^BlT (1-96+1.28)2 = M (10.5) = ZQ^ 

If we define ^ in true standard deviation units rather than observed 
standard deviation units the value C^^/r substituted for CT^enables us to 
incorporate information concerning the reliability of the dependent variable 
measure into the calculation of the sample n required to maintain a given 
statistical power. " 
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